<html>
<head><meta charset="utf-8"><title>Using const eval for compiler guarantees only · project-safe-transmute · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/index.html">project-safe-transmute</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html">Using const eval for compiler guarantees only</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="190979201"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/190979201" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#190979201">(Mar 18 2020 at 13:29)</a>:</h4>
<p>There is a desire to have some part of transmutation embedded in the compiler, as official endorsement of certain casts. The current different crates all embody different underapproximations possible casts, however there is disparity how and which underapproximations are made available within the type system. This problem is exacerbated by validity vs. safety tradeoffs: As far as I'm aware only <code>typic</code> makes a clear distinction between the two it's more involved because of it..<br>
Let's approach the problem from a different angle: The methods <code>mem::zeroed</code> and <code>mem::uninitialized</code> recently have been changed to do layout checks of the target type, and panic at runtime when they are used to definitely create an _invalid_ instance of a type (no checks for safety of course). What if there was a <code>const fn compatible_layout&lt;T, U&gt;() -&gt; bool</code> in the core language that models an RFC.-defined and SemVer compatible relation of types, i.e. an underapproximation of types for which the language wants to opt-in to such a guarantee? And additionally <code>compatible_transmute&lt;T, U&gt;</code> that panics when compatible layout does not hold, i.e. is only <code>unsafe</code> due to _safety_ invariants but not due to validity invariants. The exact set of methods and guarantees subject to discussion.<br>
This might be useful for all multiple cases. For those looking for compile time guarantees through the type system, a sound user-defined trait system safely encapsulates some underapproximation of those transmutes allowed by the standard library. If such a crate were to use <code>compatible_transmute</code> then it would be easier to audit its trait system. Even if its own logic is faulty, the worst is a panic and not undefined behaviour. Note that this would be regarded as a _bug_ in the crate interface, a correct crate should not allow unguaranteed transmutes.<br>
This would also (hopefully) fight the proliferation of private transmutes. Small crates that use only very simple internal casts might not want to depend on a large crate for the job, i.e. don't want to include a whole proc-macro machinery. What is lost in compile time guarantees can in this case be compensated with tests while the simplicity of core functions makes the whole ordeal sound. It could even be made _safe_ by only additionally requiring the <code>Transparent</code> marker trait, and no other.<br>
And lastly, it could serve as a stepping stone for more experiments. Some use cases require byte-to-type transmutes as pure optimization (e.g. in <code>image</code> decoding fills an arbitrary byte buffer but _can_ copy primitive slices directly if the buffer is correctly aligned. It's the caller's responsibility to ensure this for their byte allocation.). Here, a <code>const fn</code> would be more immediately available than depending on trait specialization for such an optimization.</p>



<a name="191006754"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191006754" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hanna Kruppe <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191006754">(Mar 18 2020 at 16:34)</a>:</h4>
<p>I'm generally a fan of compiler intrinsics that add very little magic to enable a lot of useful library code, but I'm not quite sure if the one intrinsic you describe is all that feasible and useful:</p>
<ul>
<li>"Layout" is a very overloaded term, and different operations that fall under the umbrella of this project require "compatibility" in different senses. For example, consider alignment: by-value type punning (literal <code>transmute</code>) does not care about alignment at all, while type punning behind references requires an inequality on alignment, whereas type punning of <em>owned allocations</em> (e.g. Vec&lt;T&gt; to Vec&lt;U&gt;) requires strict equality. There is a "most conservative" option across all these variants, but that might make the rules too restrictive for the intrinsic to be of much use. Multiple intrinsics might be necessary.</li>
<li>How useful is the intrinsic and the <code>compatible_transmute</code> function for implementing libraries that codify safe transmutes? It depends on how they are defined,  but you seem to suggest they should take validity into account but not safety invariants. That would not suffice to make libraries built out of <code>compatible_transmute</code>+ custom traits automatically sound. Other crates that make more precise approximations of sound transmutes (e.g., by checking runtime values) can't build on  the intrinsic either, as it will have both false negatives and false positives for their purposes.</li>
</ul>



<a name="191046207"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191046207" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191046207">(Mar 18 2020 at 21:43)</a>:</h4>
<p>There are of course some open questions to answer but I gave them some consideration. 'T is  layout compatible with U' should be interpreted as 'all valid bit-patterns of T are valid bit-patterns of U' in the use above.</p>
<ol>
<li>Casting of references needs to be considered separately from transmutes in any case, even more so for a trait solution. There is an overlap between the two but neither is contained in the other. Consider that the former does not require <code>Sized</code> of any of its arguments (requireing the unsized locals feature sounds suboptimal), and that references are fundamental (which complicates trait impls for them). On the other side, transmuting of <code>Cell&lt;&amp;'static T&gt;</code> to <code>Cell&lt;&amp;'a T&gt;</code> is sound but doing so behind any kind of references is not. A trait to mark types without any safety invariants contains a subset of the overlap though, I think? (The  <a href="https://github.com/rust-secure-code/mem-markers/issues/6" target="_blank" title="https://github.com/rust-secure-code/mem-markers/issues/6"><code>Transparent</code></a> trait might be a good candidate, but it's a bit of a misnomer.) Anyways, this seems to hint that solving all three kinds of casts at the same time is more complicated than considering them separately. And maybe that <code>std</code> marker-trait would help, as they permit overlapping impls more easily. That however, is yet another unstable feature and we might be better off with initial implementations that don't create such inter-dependencies.</li>
<li>I consider the intrinsic highly useful if the crates have the concept of <code>Transparent</code> which appears to be a common choice anyways. <code>zerocopy</code>. <code>bytemuck</code>, <code>plain</code>, <code>typic</code> all require it for their safe cast methods—although only the last makes it an explicit trait. As for runtime safey check—by the way, how should we term such code, to not allude to validity—that is a separate issue to consider but I'm quite confident that this can be done as an independent additional follow up with a wrapper. In similar ways as <code>MaybeUninit</code> protects an instance with not-yet-fulfilled validity invariants there could be wrapper to protect one with not-yet-checked safety invariants. Crates that want to build safety checks prior to a standardized wrapper, or some custom safety checking code flow, can still make use of <code>compatible_transmute</code>. It does not eliminate the proof obligations but removes a large part of them. The remaining obligations also do not depend on the Rust language—except for those of types provided in <code>core</code>. As opposed to invalid values, it is not instant UB to make a reference to the not-yet-safety-checked instance, it only might be UB to use any of its methods that could rely on custom invariants.</li>
</ol>



<a name="191050762"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191050762" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lokathor <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191050762">(Mar 18 2020 at 22:28)</a>:</h4>
<p>MaybeUninit is <em>also</em> the correct wrapper for incomplete safety</p>



<a name="191060589"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191060589" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191060589">(Mar 19 2020 at 00:31)</a>:</h4>
<p>I disagree, it's weaker than necessary. In particular, it does not contain any niches of the type. The most concrete wrapper would be a completely opaque <code>ManuallyDrop&lt;T&gt;</code>-like that doesn't provide <code>Deref</code> or any other safe access to the instance. This allows the compiler to assert that value has the layout of the type with all its niches and padding bytes but does not require any safety invariants to hold.</p>



<a name="191060966"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191060966" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191060966">(Mar 19 2020 at 00:39)</a>:</h4>
<p>Litmus test: Such a wrapper would allow sound transmutes between itself and <code>T</code> in both directions, meanwhile MaybeUninit would not.</p>



<a name="191086474"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191086474" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lokathor <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191086474">(Mar 19 2020 at 09:19)</a>:</h4>
<p>If the wrapper is specifically "safety not enforced", then how can you soundly remove it so easily?</p>



<a name="191094861"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191094861" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191094861">(Mar 19 2020 at 10:39)</a>:</h4>
<p><code>unsafe fn into_inner</code>, and I said the transmute is <em>sound</em>, not <em>safe</em>.</p>



<a name="191095939"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191095939" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hanna Kruppe <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191095939">(Mar 19 2020 at 10:47)</a>:</h4>
<p>That is an, uh, <em>unusual</em> use of the term "sound". Quoting UGC glossary:</p>
<blockquote>
<p>Accordingly, we say that a library (or an individual function) is sound if it is impossible for safe code to cause Undefined Behavior using its public API. Conversely, the library/function is unsound if safe code can cause Undefined Behavior.</p>
</blockquote>



<a name="191096547"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191096547" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191096547">(Mar 19 2020 at 10:52)</a>:</h4>
<p>It's the use of <code>typic</code>, I merely used it</p>



<a name="191096669"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191096669" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191096669">(Mar 19 2020 at 10:54)</a>:</h4>
<p>That said, I agree it's not named in an optimal way (as also the <code>Transparent</code> trait). We should create a thread to agree on such terminology.</p>



<a name="191096842"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191096842" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191096842">(Mar 19 2020 at 10:55)</a>:</h4>
<p>As <code>transmute</code> is <code>unsafe</code>, I don't think the quote from the UGC applies.</p>



<a name="191098049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191098049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hanna Kruppe <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191098049">(Mar 19 2020 at 11:07)</a>:</h4>
<p><span class="user-mention silent" data-user-id="229913">HeroicKatora</span> <a href="#narrow/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only/near/191046207" title="#narrow/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only/near/191046207">said</a>:</p>
<blockquote>
<ol>
<li>Casting of references needs to be considered separately from transmutes in any case, even more so for a trait solution. [...] </li>
</ol>
</blockquote>
<p>Yes, they are different operations, but both are strongly desired and there is enough interplay between them that treating them as entirely separate problems might result in a worse set of solutions overall.</p>
<blockquote>
<p>As for runtime safey check—by the way, how should we term such code, to not allude to validity—that is a separate issue to consider but I'm quite confident that this can be done as an independent additional follow up with a wrapper. In similar ways as <code>MaybeUninit</code> [...]</p>
</blockquote>
<p>Validity does play a role too. For example, if the destination type contains a bool or enums, you to check that the source value corresponds to a valid disciminant. If you're transmuting a reference, you might need to check its alignment (in addition to whatever checks you need w.r.t. the referent). I don't know how to simplify such checks with the kind of wrapper you propose. In general you need field offsets and access raw bytes.</p>



<a name="191098840"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191098840" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191098840">(Mar 19 2020 at 11:16)</a>:</h4>
<p>Both of your examples are validity checks and not safety checks. But in a pure safety check you could rely on valid initialization. It would be allowed to (unsafely) load fields of a struct for example, or even to match on an enum's discriminant. You only need to be careful not to use any of the type's methods and impls that may rely on its safety invariants (e.g. by doing recursive safety checks on fields first, or only having <code>Transparent</code> fields).</p>



<a name="191098981"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191098981" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> HeroicKatora <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191098981">(Mar 19 2020 at 11:17)</a>:</h4>
<p>Your comment does make a point though, that even validity checks might be dynamic.</p>



<a name="191302636"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Using%20const%20eval%20for%20compiler%20guarantees%20only/near/191302636" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lokathor <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Using.20const.20eval.20for.20compiler.20guarantees.20only.html#191302636">(Mar 20 2020 at 21:07)</a>:</h4>
<p>Well an example safety check would be the len field of some sort of vec-like thing</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>